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Abstract 


In this work, we have e\aluated the performance of continuous digit rec- 
ognizer on two most popular!} used front-end features, namely LPC- and 
MEL-cepstral features so as to study the robustness of the above said fea- 
tures for digit recognition task For this purpose we have implemented a 
HMM based continuous digit recognizer using development environment pro- 
vided by CSLU-Toolkit Our study finds that MEL-cepstral feature provides 
slightly better word and sentence level accuracies compared to LPC-cepstral 
features Our study also indicates that for a fixed product of number of states 
and number of mixtures per state m a model, the models with higher number 
of state result m better word as well as sentence level accuracy In the later 
part of this work, we have reported a study on the relationship between any 
two speakers which is important from the point of view of gaming an insight 
into the development of speaker-independent speech recognition systems 
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Chapter 1 
Introduction 


Speech Recognition has a long history of being one of the difficult problems 
in Artificial Intelligence and Computer Science Work in speech recognition 
predates the invention of computers However, the serious work in speech 
recognition started in the late fifties with the advent of digital computers 
equipped with A/D converters The problems of segmentation, classification, 
and pattern matching were explored m the sixties and a small vocabulary 
connected speech task was demonstrated In the early seventies, the role 
of syntax and semantics m connected speech recognition was explored and 
demonstrated The seventies also witnessed the development of a number 
of basic techniques such as dynamic time warping, network representation, 
probabilistic modeling, beam search, and forward-backward algorithm The 
early eighties witnessed a trend towards practical systems with very large 
vocabularies but computational and accuracy limitation made it necessary to 
require one-word-at-a-time with a short pause between them In nineties, the 
emphasis shifted to natural language front ends to the recognizer and task 
shifted to commercial application specific ones At the same time, speech 
recognition technology has been increasingly used within telephone networks 
to automate as well as enhance operator services 

Research in speech recognition has followed two primary roots those 
adopting a knowledge-based approach, and those adopting a statistically 
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data-based approach Knowledge-based approach to speech recognition and 
understanding [11] have attempted to express human knowledge of speech 
in terms of acoustic-phonetic rules based on specific feature of the acoustic 
waveform For these approaches, the acoustic signal is first segmented and la- 
belled into phoneme-hke units, and then resulting phoneme string is used for 
lexical and syntactic analysis Words in the lexicon are represented in terms 
of the phonemic spellings and syntax is usually described by conventional 
linguistic means It is known that a human can be trained to read speech 
spectra which supports the existence of distinct features in speech spectrum 
[27], but the machine realization of this human ability is far poorer than the 
well trained human expert The possible reasons for the poor performance of 
this approach are the absence of good understanding of the human auditory 
mechanism and the inability of human experts to formalize comjiletely their 
knowledge Thus speech recognition remains an unsolved problem for the 
knowledge-based approach 

In contrast to the knowledge-based approach, the data-based statistical 
approaches have achieved considerable success. These are based on modeling 
the speech signal by itself by some well-defined statistical algorithms that 
can automatically extract knowledge from speech data The predominant 
class of these algorithms is the hidden Markov model (HMM) [7, 13] An 
HMM-based system depends on three key factors 

1 a detailed modeling scheme which is capable of accommodating various 
uncertainties, 

2 access to sufficient speech data, and 

3 an automatic learning algorithm to improve the recognition accuracy 

By using HMMs, the speech signal variability m parameter space and 
time can be modeled effectively Unlike the knowledge-based approach, the 
HMM learning procedure is achieved by presenting the speech data to HMMs 
and automatically improving the models by data as opposed to some heuristic 


2 



rules presented by human experts In general, the more data presented to the 
model, the hig ler the recognition accuracy achieved HMM methods have 
presented speech recognition with a sound theoretical basis, and have resulted 
m significant advances in large-vocabulary continuous speaker-independent 
speech recognition [13] 

The HMM can be based on either discrete output probability distribu- 
tions or continuous output probability density functions, which are veiy^ im- 
portant to acoustic modeling Both discrete HMM and continuous HMM are 
widely used in state-of-the-art speech recognition systems [13, 17, 21] For 
the discrete HMM, vector quantization (VQ) makes it possible to use a non- 
parametric, discrete output probability distribution to model the observed 
speech signals The objective of VQ is to find the set of reproduction vectors, 
or codebook, that represents an information source with minimum distortion 
Under discrete HMM framework, VQ is first used to obtain discrete output 
symbols The discrete HMM then models observed discrete symbols In con- 
trast, the continuous mixture HMM [20] uses continuous mixture probability 
density functions to model speech parameters directly without using VQ, and 
usually needs extensive training data and computation times 

On the other hand, the semi-continuous HMM [7] which is a very general 
model including both discrete and continuous mixture HMMs as its special 
forms, unifies VQ, the discrete HMM, and the continuous mixture HMM 


1.1 Scope and Organization of the Thesis 

In speech recognition systems, researchers first of all used linear-prediction 
based features, later the use of Mel-cepstral features reportedly improved the 
recognition accuracy We have tried to find out m this work how the choice of 
a particular front-end feature is related to the number of states and number 
of mixtures per state in the model used in the recognition system Such a 
study is of importance as it reveals the robustness of a particular feature m 
HMM based recognition systems 
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In this thesis, we w'lll provide the background information on the hidden 
Markov modeling of speech in Chapter 2 In the same chapter we will describe 
how HMMs are used for speech recognition 

In Chapter 3, we will describe the various front-end features used in 
our study In Chapter 4, we will describe the database used for this study 
and the recognizer implementation Chapter 5 contains the summary of our 
experimental results and the conclusion 

Finally, m Chapter 6, we will present an additional study about the re- 
lationship betw'een speakers which could to provide an important clue in 
realization of speaker independent speech recognition systems 



Chapter 2 

Hidden Markov Models 


2.1 Definition of HMM 

The Hidden Markov Model is a finite set of states, each of which is associated 
with (generally multidimensional) probability distribution [19] Transitions 
among states are governed by the set of probabilities called transition prob- 
abilities In a particular state the outcome or observation can be generated, 
according to the associated probability distribution It is only the outcome, 
not the state is visible to an external observer and therefore the states are 
“hidden” to the outside hence the name Hidden Marko\ Model (HMM) 

In order to define an HMM completely, following elements are needed 

• The number of states of the model, N 

• The number of observation symbols m alphabet, M If the observation 

IS continuous then M is infinite 

• A set of state transition probabilities, A = {a,j} 

az] = P{Qt+i = J 

where qt denoted the current state 

Transition probabilities should satisfy the normal stochastic constraints, 

> 0 , I < 1,3 < N 
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and 


N 

a,j = 1, 1 <i < N 

j=i 

• A probability distribution in each state, B = {bj{k)}, 

bj{k) = P{ot = Vk\qt=j}, 1<J<N, l<k<M 

where Vk denotes the k^^ observation symbol in the alphabet, and Ot 

the current parameter vector 

Following stochastic constraints must be satisfied, 

bj(k) >0, 1<J<N, l<k<M 

and 

M 

= 1 <}<N 

k=l 

If the observation are continuous then we will have to use a continuous 
probability density function, instead of a set of discrete probabilities In 
this case we specify the parameters of the probability density function 
Usually the probability density is approximated by a weighted sum of 
M Gaussian distiibution H, 

M 

m=l 

where, 

= weighting coefficient 
fj,jm = mean vectors 
Sjm = covariance matrices 

Cjm should satisfy the stochastic constraints, 

Cjm>0, l<j<N,l<m<M 
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• The initial state distribution, tt = {tTi} 
where, 

TT, = P{qt = ^}, 1 < i 

Therefore we can use the compact notation 

A = (A,B,7r) 

to denote an HMM with discrete probability distributions, while 

A — (A, Cjfji, fJ’jmt ^jmi 
to denote one with continuous densities 


2.2 Assumptions in The Theory of HMMs 

For the sake of mathematical and computational tractability following as- 
sumptions are made in the theory of HMMs 

1 The Markov Assumption As given m the definition of HMMs, the 
transition probabilities are defined as, 


a,j = P{qHi =j\qt = i} 


In other words it is assumed that the next state is dependent only 
upon the present state This is called the Markov assumption and the 
resulting model becomes actually a first order HMM However generally 
the next state may be dependent on past k states and it is possible 
to obtain such model, called order HMM But it is seen that the 
a higher order HMM will have higher complexity Even though first 
order HMMs are the most common, some attempts have been made to 
use higher order HMMs also 
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2 The Stationary Assumption Here it is assumed that the state tran- 
sition probabilities are independent of the actual time at which the 
transition takes place Mathematically, 

J I gti = = P{qt2+i = J I 9t2 = 0 

for any ti and t2 

3 The Output Independence Assumption This is the assumption that the 
current output (observation) is statistically independent of the previous 
outputs (observations) We can formulate this assumption mathemat- 
ically, by considering a sequence of observations, 

0 = 01,02,03, jOr 

Then according to the assumption for an HMM A, 

T 

P{0\qi,q2, ,gr,A} = 

<=i 

However unlike other two, this assumptions has a very limited validity 
In some cases this assumption may not be fair enough and becomes a 
severe weakness of HMMs 

2.3 The Three Basic Problems for HMMs 

Once we have the HMM, three basic problem of interest must be solved for 
the model to be useful for the real-world applications 

1 The Evaluation Problem Given an HMM A and a sequence of ob- 
servations O = 0i,02, ,Ot 1 what IS the probability that the given 
observation sequence is generated by the model, P {0 | A}*^ 

2 The Decoding Problem Given a model A and the sequence of obser\'a- 
tions O = Oi, 02, . ,ot, what is the most likely state sequence in the 
model that produced the observations'^ 
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3 The Learning Problem Given the model A and a sequence of obsen^a- 
tions, how should the parameter of the model (A, B, tt) be adjusted in 
order to maximize P{0 | A}*^ 

Evaluation problem can be used for the isolated (word) recognition Decod- 
ing problem is related to the continuous recognition as well as the segmen- 
tation Learning problem must be solved, if we want to tram an HMM for 
the subsequent use of recognition tasks 

2.3.1 The Evaluation Problem and Forward Algorithm 

We have a model A = (A, P, tt) and a sequence of observations O = Oi, ,ot 
and P(0 I A) must be found We can compute this quantity using simple 
probabilistic arguments But this computation involves number of operations 
of the order of N'^ This is very large even if the length of the sequence, T, 
IS moderate Therefore we use another method for this computation which 
has a considerably low complexity and makes use an auxiliary variable 
called the forward variable The forward variable is defined as the probability 
of the partial observation sequence 01 , 02 , ,Ot, when it terminates at the 
state i Mathematically, 

at(i) = P{oi,02, ,Ot,gf = i|A} (2 1) 

then it is easy to see that following recursive relationship holds, 

N 

Qt+iO) =bj(ot+i)J2^t(i)a^j, 1<J<N, l<t<T-l (2 2) 

t=i 

where, 

ai(j) = 1<J<N 

Using this recursion we can compute, ar(z), 1 < i < N and then the 
required probability is given by, 

P{0|A}=f;a,.(.) (2 3) 

1=1 
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The complexity of this method, known as forward algorithm is proportional to 
N^T, which is linear with respect to T whereas direct computation mentioned 
earlier, had an exponential complexity 

In a similar way we can define the backward variable as the probability 
of the partial observation sequence 04+1,04+2, ,ox, given that the current 
state IS i Mathematically, 

Pt{t) = P{o4+i, 04+2, , Or, ft = 2 1 A} (2 4) 

As in the case of at(i) there is a recursive relationship which can be used to 
compute j3t{i) efficiently, 

N 

= l<t<N,l<t<T-l (25) 

where. 


Prii) = 1, l<t<N 


Further we can see that, 

«i(0A(j) = P{0, ft = I I A}, 1 < 2 < A, 1 < t < T (2 6) 

Therefore this gives another way to compute p{0 | A}, by using both forward 
and backward variables as given below, 

^•{0 I A} = f;p{0,9, = I I A} = f:a,(.)AW (2 7) 

1=1 t=l 

2.3.2 The Decoding Problem and Viterbi Algorithm 

In this case we want find the most likely state sequence for a given sequence 
of observations O = 01,02, . ,or and the model A = (A,S,7r) The solu- 
tion to this problem depends upon the way “most likely state sequence” is 
defined One approach is to find the most likely state ft at time t = t and to 
concatenate all such q'^s But some time this method does not give physically 
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meaningful state sequence Therefore we would go for another method which 
has no such problem In this method, commonly known as Viterbi algorithm 
[5, 24], the whole state sequence with the maximum likelihood is found In 
order to facilitate the computations we define an auxiliary variable. 




which gives the highest probability that partial observation sequence and 
state sequence can have,when the current state is i It is easy to observe that 
the following recursive relationship holds 


= bj{ot+i) 


max StU)a„ 


l<^<N,l <t<T-l 


(2 9) 


where, 


5i(z) = 7rj6j(oi), 1 < I < N 

So the procedure to find the most likely state sequence starts from the 
calculation of Srij), 1 < j < N using the above recursion, while always 
keeping a pointer to the “winning state” in the maximum finding operation 
Finally state j*, is found where 

= arg m^^(5r0), (2 10) 

and starting from this state, the sequence of the states is back traced as the 
pointer in each state indices This gives the required set of states This whole 
algorithm can be interpreted as a search m a graph whose nodes are formed 
by the states of the HMM in each of the time instants t, 1 <t<T 


2.3.3 The Learning Problem 

Generally, the learning problem is to adjust the HMM parameters, so that 
the given set of observations (called the training set) are represented bj the 
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model in best way for the intended application Thus it would be clear 
that the “quantity” we wish to optimize will be different from application to 
application In other there may be several opUmtzahon cntena for learning, 
out of which suitable one is selected depending on the application 

There are two main optimization criteria found m ASR literature, Max- 
imum Likelihood (ML) and Maximum Mutual Information(MMI)[15] Since 
we have used ML optimization criterion m our experiments so the solution to 
the learning problem under this criterion is described in detail the following 
sections, while only basic principle is discussed for MMI criterion 

Maximum Likelihood(ML) Criterion 

In ML criterion we try to maximize the probability of a given sequence of 
observation O’", belonging to a class w, given the HMM of the class w, 
with respect to the parameters of the model This probability is the total 
likelihood of the observations and can be expressed mathematically as, 

Ptot = P{0- I A«} (2 11) 

However we consider one class only at a time we can drop super and subscript 
‘m’ Then the ML criterion can be written as 


Ptot=P{0\X} (2 12) 

However there is no knovm way to solve this analytically for the model 
A = (A, B, tt), which maximizes the quantity Ptot But we can choose model 
parameters such that it is locally maximized, using a iterative procedure, like 
Baum- Welch method or a gradient based method 

Baum- Welch Algorithm This method can be derived using simple “occur- 
rence counting” arguments or using calculus to maximize the auxiliary quan- 
tity called Q-function, defined as, 

J) = 'LP{0,q I A) logP(0,« I A) (2-13) 
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over A Because 


Q{\ A) > QiX, A) => P(0 I A) > P(0 I A) (2 14) 

we can maximize the function Q(A,A) over A to improve A m the sense 
of increasing the likelihood P(0 | A) Eventually the likelihood function 
converges to a critical point if we iterate the procedure [2] 

To describe the Baum- Welch algorithm, (also known as Forward-Backward 
algorithm), we need to define two more variables, in addition to the forward 
and backward variables defined in prenous section These variables can how- 
ever be expressed in terms of forward and backward variables 


First one of those variables is defined as probability of being in state i at 
t = t and 111 state j at t = t + 1 Formally 


itihj) =P{qt = i,qt+\ =j\o,x) 

(2 15) 

This IS same as, 

^ P{qi = hqt+i=j,o \ X) 

P(0|A) 

(2 16) 

Using the forward and backward variables this can be expressed 

as, 


(217) 

The second variable is a-posteriori probability. 


7t(z) = P(gt = i lO,A) 

(2 18) 

that IS the probability of being in the state i At t = t, given the observation 
sequence and the model In forward and backward variables this can be 

expressed as, 

(219) 


One can see that the relation between 7 t(i) and ^t(i, j) is given by, 


7tW=i;6(bj), \<i<N,l<t<M (2 20) 

3=1 
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Now it IS possible to describe the Baum- Welch learning process, where the 
parameters of HMM is updated in such a way to maximize the quantity 
P(0 I A) Assuming the starting model A = (A, J5,7r), we calculate the 
a's and /3's using the recursions (2 2) and (2 5), and then ^'s and 7's using 
(2 17) and (2 20) Next step is to update the HMM parameter according to 
Eq (2 21) to (2 23), known as re- estimation formulas 

^» = 7iW, l<i<N (2 21) 

(2 22 ) 

(2 23) 


a, 


V 




EL 7.0) 
EL 7.0)’ 


b,{k) = 




l<i,J <N 


These re-estimation formulas can be easily modified to deal with the con- 
tinuous density case too and the re-estimation formulas for the coefficient of 
the mixture density, 1 e , /.jjt, and Ej*:, are of the form [19], 


_ _ E^I 7.0. fc) Ot 

Er=.7.o,t) 

V - E^i 7.0. (o. - - f^jk)' 

E^i 7.0. k) 


(2 24) 
(2 25) 
(2 26) 


where prime denotes vector transpose and where 7t(j, k) is the probability of 
being in the state j at time t with the mixture accounting for Ot, 1 e , 


7.0. 


a.0)A0) 


.EjL 



(2 27) 
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Gradient Based Method In gradient based method, any parameter © of 
the HMM A is updated according to standard formula, 


- T] 


dJ 

dQ 


(2 28) 


e=e°'<' 


where J is the quantity to be minimized We define in this case. 


J=E,a - -log(P{0 I A}) = - log(Ptot) 


(2 29) 


Since minimization of J = Eml is equivalent to the maximization of Ptot, 
Eq (2 28) yield the required ML optimization criterion The derivative 
for any parameter © of the model can be obtained by relating J to the model 
parameters via Ltot As a key step to do so, using Eqs (2 7) and (2 12) we 
can obtain, 

Pto, = 1 1 A} = i:o,WA(>) (2 30) 

t=l »=1 

Differentiating the last quantity in Eq (2 29) with respect to an arbitrary 
parameter ©, 

dJ ^ 1 dPtoi 

dQ 


(2 31) 


Plot dQ 

Eq (2 31) gives if we know which can be found using Eq (2 30) Since 

there are two mam parameter sets in HMM, namely transition probabilities 
Oij, 1 < 1,7 < and observation probabilities bj{k), 1 < j < N,1 < k < M, 
we can find the derivative for each of the parametei sets and hence the 
gradient |g 

To find the gradient with respect to transition probabilities using the 
chain rule, 

dPtot ^ dPtot atO) 


da. 


ij 


T 


By differentiating Eq (2 30) with respect to 0(4(7) we get, 

dPtot 


dott{3) 


= A(7) 


(2 32) 


(2 33) 
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and differentiating a time shifted version of Eq (2 2) with respect to a,j, 


= (2 34) 

Eqs ( 2 32), ( 2 33)and( 2 34) give, and substituting this quantity in 

Eq (2 31), we get the required result 


dj 


da, 


V 


Ptot (=1 




(2 35) 


Similarly to find the gradient with respect to observation probabilities 
again using the chain rule. 


dPt 


tot 


dPtot oit(j) 


dbj{ot) dat{j) dbj{ot) 

Differentiating a time shifted version of Eq (2 2) with respect to bj(ot) 

datO) ^ atjj) 

dbj{oi) bj{ot) 


(2 36) 


(2 37) 


Finally we get the required probability, b> substituting ■§r^ m Eq (2 31), 
which IS obtained by substituting Eqs (2 37) and (2 33) m Eq (2 36) 


dJ 


dbj{ot) 


1 

bjiot) 


tot 


(2 38) 


Usually this equation is given the following form, by first substituting for Ptot 
from Eq (2 30) and then substituting from Eq (2 20) 


dJ ^ 7t(j) 

dbj(ot) bj(ot) 


(2 39) 


If the continuous densities are used then and can be found by 

further propagating the derivative using the chain rule 


16 



Maximum Mutual Information(MMI) Criterion 

In ML criterion we optimize an HMM for onh one class at a time, and 
do not touch the HMMs for other class at that time This procedure does 
not involve the concept of “discrimination” which plays an important role 
in Pattern Recognition and of course in ASR Thus ML learning procedure 
gives a poor discrimination ability to HMM system, specially when the esti- 
mated parameters (in training phase) do not match with speech inputs used 
m recognition phase This type of mismatch can arise due to two reasons 
One the training and recognition data have considerably different statistical 
properties, and the other is the difficulties in obtaining reliable parameters 
estimates m the training The MMI criterion on the other hand considers 
HMMs of all classes simultaneously, during the training Parameters of the 
correct model are updated to enhance it’s contribution to the observations, 
while parameters of the alternative models are updated to reduce their con- 
tributions This procedure provides a high discriminative ability to the HMM 
system and thus MMI belongs to so called “discriminative training” category 


2.4 Use of HMMs in Isolated Recognition 

Isolated recognition in general is based on any kind of isolated speech unit, 
which can be word, a sub-word or even a concatenation of words However 
only isolated word recognition has direct practical applications, while the iso- 
lated recognition based on other two types of units has basically theoretical 
value Specially the sub-word recognition in isolated mode can give a good 
indication about the continuous recognition based on the same techniques 
In a simple isolated speech unit recognition, where the vocabulary con- 
tains V speech units, we can use the system depicted in Figure 2 1 There 
are however many possible ways to provide solution of the task, because 
there are many optimality criteria, and several algonthmic implementation 
of a particular criterion Out of those, only Baum-Welch method, as the one 
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used in implementation, is described below Our task is comprised of two 
sub tasks, namely the training and the recognition 



Figure 2 1 Block diagram of an isolated word HMM recognizer (after Rabiner 
[20]) 


2.4.1 Training 

The training procedure can be summarized as follows: 

1 Initialize each HMM, A, = (A,,jB,,7rt), 1 < I < N with values gen- 
erated randomly or using an initialization algorithm like segmental k- 
means [8] 
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2 Take an observation sequence and 

• calculate the forward and backward probabilities for each HMM, 
using Eqs (2 2) and (2 5) 

• using Eq (2 17) and (2 20), calculate the auxiliary variables ^ and 
7 

• update parameters in each model, using Ekjs (2 21), (2 22) and 
(2 23) 

3 Go to step 2, unless all observation sequences are considered 

4 Repeat step 2 to 3 until a convergence criteria is satisfied 

This procedure can be easily modified for the continuous density case, by 
using the Eqs (2 24), (2 25) and (2 26) to reestimate the coefficients of the 
mixture density instead of Eq (2 23) m step 4 of above training procedure 

2.4.2 Recognition 

Comparative to the training the recognition is much simpler and the proce- 
dure IS given below 

1 Take the observation sequence to be recognized and 

• calculate the forward and backward probabilities for each of HMM 
using the recursion (2 2) and (2 5) 

• using Eq (2 12) calculate the likelihood, 

l<m<M 

2 The recognized class w*, to which the observation sequence belongs, is 
given by 

w* = arg max FJf. 

^ \<m<M 
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3 Go to step 2, unless all observation sequence to be recognized are con- 
sidered 

The recognition rate in this case can be calculated as the ratio of the cor- 
rectly recognized speech units and the total number of speech units (in the 
observation sequence) to be recognized 

2.5 Use of HMM in Continuous Recognition 

In isolated recognition we have used one HMM for each speech unit But 
in continuous recognition, this is not possible because a continuous sequence 
of speech units, which is usually called a sentence, is to be recognized and 
hence the number of possible sentences may be quite high even for a small 
vocabulary In addition there are other two fundamental problems associated 
with continuous recognition 

1 We do not know the end points of the speech units contained in the 
sentence 

2 We do not know how many speech units are contained in the sentence 

Because of the problems mentions above, the continuous recognition is more 
complicated than isolated recognition However HMMs a provide good frame 
work for continuous recognition also. In this case we connect the HMMs for 
each speech units m a sentence to form a large HMM m a manner shown 
in Figure 2 2 This representation is called clamped grammar case HMM 
in the free grammar case is obtained by connecting all speech units in the 
vocabulary as shown in Figure 2 3 The transitions between the speech units 
are derived using so called language model 

A block diagram of a canonic system for connected digit recognition is 
shown in Figure 2 4 There are three basic steps in the recognition process, 
namely 
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Figure 2 2 Sentence HMM in clamped grammar case shown with 4 speech 
units 



Figure 2 3 Sentence HMM m free grammar case shown with 4 speech units 


1 Spectral analysis, in which the speech signal, s(n), is converted to an 
appropriate spectral representation, e g , filter-bank vector, LPC-based 
vector 

2 Connected word pattern matching, in which the sequence of spectral 
vectors of the unknown (test) connected digit string is matched against 
whole word (single digit) patterns using any of the algorithms, viz , 
Level building algorithm, Two-level DP algorithm etc The output of 
this process is a set of candidate digit strings, generally of different 
lengths, ordered by distance (likelihood, probability) score 

3. Postprocessing, in which the candidate digit strings are subjected to 
further processing (based on digit duration, word stress, etc ) so as 
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to eliminate unreasonable (unlikely) candidates The postprocessor 
chooses the most likely digit string from the ordered list of candidates 
which passed the postprocessor tests 


SINGLE 
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Figure 2 4 Block diagram of connected digit recognition process 


2.5.1 Statistical Language Model 

The goal of language model, as already mentioned, is to estimate the prob- 
ability of a sequence of speech units Assume that w is a specified sequence 
of L speech units, 

W = U>i,W2, 

were interested m the probability, P{w} This can be approximated by, 

which IS called an N-gram language model These probabilities can be evalu- 
ated by occurrence counting in the database However it is clearly impractical 
except for the case of N=2 or at the most N=3 In the case of N=2, we call it 
higram language model and it is the most widely used case [19] An extreme 
case of bigram model called word pair model can be obtain by quantizing 
the bigram probabilities between 0 and 1 But the most relaxed grammar is 
710 grammar ■where any speech unit can be follo'wed every speech imit m the 
vocabularj' 
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2.5.2 Training 

The procedure of training the model using the gradient based ML criterion 
IS summarized as below 

1 Initialize each HMM Ai = 1 < % < N with values gen- 

erated randomly or using an initialization algorithm like segmental k- 
means 

2 Take the observation sequence of a sentence and 

• Form the corresponding sentence model using the HMMs of the 
speech units contained in the sentence 

• Calculate the forward and backward probabilities using the recur- 
sion ( 2 2) and (2 5) 

• Using the Eq (2 30) calculate the likelihood of the observations in 
the sentence model 

• Using Eqs (2 35) and (2 38) calculate the gradient with respect to 
all parameters in sentence model 

• Update all parameters m each HMMs of the sentence model using 
Eq (2 28) 

3 Go to step 2, unless all observation sequences are considered 

4 Repeat step 2 and 3, until a convergence criterion is satisfied 

2.5.3 Recognition 

In the recognition phase it is assumed that trained HMMs for each of the 
speech units in the vocabulary is available The task is to find the underlying 
speech unit sequence, given an observation sequence O corresponding to an 
unknown sentence Mathematically this operation can be expressed as, 

w* = arp n^ P{w, O I A) (2 40) 
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where w = wi,W 2 , ,1^5 is an arbitrary speech unit sequence of arbitrary 
length S Since P{w} is provided by the statistical language model, so to 
find w* we have to only calculate P{0 | w, A} for every possible w It 
IS obvious that this procedure will be computationally very expensive, so a 
cheaper solution is to approximate the procedure, by finding the most likely 
state sequence, q* m the language model A, instead of speech unit sequence 
w* Formally 

q* = ar^ m^ P{q, O I A} (2 41) 

Then it is possible to trace for the corresponding speech unit sequence, via 
the state sequence In order to calculate q* we can use the Viterbi algorithm 
directly, or a method called Level butldmg [12, 21], a variant of Viterbi al- 
gorithm Since Viterbi based recognition is suboptimal, unless each speech 
unit IS corresponding to a HMM state, some attempts have been made to 
develop the efficient method to find the sentence likelihood The so called 
N-best algorithm is one of this 

Viterbi-based Search 

The Viterbi score, can be computed for all the states in the language 
model A at time t — t and then can advanced to the time instant t = t + l, 
in an inductive manner, as formulated in Eq (2 9) This procedure is known 
as time synchronous Viterbi search because it completely processes at time 
t before going into the time t = t + l Finally a backtracking pass gives the 
required state sequence However even Viterbi search can be very cumber- 
some if the number of states is large In such cases instead of keeping all 
candidates at every time instant t, only first T candidates with highest scores 
are retained This method of pruning search space is known as beam search 

Level Building 

In level building, unlike in Viterbi search, HMM for each speech unit is con- 
sidered separately The search is performed at various levels for each of 
HMM using Viterbi aigonthm, where level corresponds to the position of 
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speech unit within the possible sentence After searching at each level, we 
find the maximum score for all speech unit models for every time frame t 
The search in the next level starts with the winning score of the previous 
level at previous time frame After having completed the searches in I levels, 
the sequence of winning speech unit model at levels 1,2, ,l in that order 
represents the possible sentence which is / speech units long In order to find 
the optimum sentence a maximization is done over I 

N-best Search 

N-best search algorithm is very similar to the time synchronous Viterbi 
search. Since the purpose of N-best search is to find theoptimum speech unit 
sequence instead of optimum state sequence, a summing operation should be 
done instead of maximum finding operation However if we completely drop 
the maximum finding operation it will become forward algorithm and we go 
back again to the start Therefore the pruning is performed at each state, 
m addition to the pruning of the beam, keeping only the first N paths with 
the highest scores This algorithm also does not give theoretically optimum 
sentence. At the end the algorithm gives N most likely sentences, and for 
simple task N=1 is enough. 

Computing Error Rate 

Since we do not know how many speech units are there in the sentence to be 
recognized, the result of the recognition can have less or more speech units 
than the correct sentence Therefore the recognized sentence is matched 
with the correct sentence with help of a dynamic programming algorithm 
In continuous recognition case, there are three types of errors substitution, 
deletion, and insertion After finding the percentage word correctly recog- 
nized, the insertions, deletions, and substitutions The word accuracy is 
computed from these as follows 

Word Accuracy = % Word correctly recognised — % Insertions 
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Chapter 3 

Front-End Features 


The various speech recognizer developed till date can be classified into differ- 
ent classes depending on the varying emphasis to different algorithmic inputs 
to this field from a wide variety of disciplines, including statistical pattern 
recognition, communication theory, signal processing, combinatorial mathe- 
matics, and linguistics, among others But the most common denominator 
of all recognition systems is the signal-processing front-end, which converts 
the speech waveform to some type of parametric representation (generally at 
a considerably lower information rate) for further analysis and processing 

A wide range of techniques exists for parametrically representing the 
speech signal These include the short time energ}', zero crossing rates, level 
crossing rates, and other related parameters Probably the most impor- 
tant parametric representation of speech is the short time spectral envelope 
Spectral analysis methods are therefore generally considered as the core of 
the signal-processing front-end in a speech recognition system There are 
two dominant methods of spectral analysis-namely, the bank-of-filter spec- 
trum analysis model, and the linear predictive coding (LPC) spectral analysis 
model 

The overall structure of the bank-of-filter model is shown in Figure 3 1 
The speech signal, s(n), is passed through a bank of Q bandpass filters whose 
coverage spans the frequency range of interest in the signal The individual 
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filters can and generally do overlap in frequency domain, as shoi^n at bottom 
of Figure 3 1 The output of the bandpass filter,Xn(exp^‘^‘) (where tu, is the 
normalized frequency 2'nftfFs, with Fg the sampling frequency) is the short- 
time spectral representation of the signal s(n), at time n, as seen through 
the bandpass filter with center frequency Ut It can be easily seen that 
in the bank-of-filter model each bandpass filter processes the speech signal 
independently to produce the spectral representation Xn 
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Figure 3 1 Bank-of-filters analysis model 


The LPC analysis approach, as illustrated m Figure 3 2, performs spec- 
tral analysis on blocks of speech (speech frames) with an aU-pole modeling 
constraint This means that the resulting spectral representation X„(expJ") 
is constrained to be of the form a/^(exp^‘^), where j4(exp>^ ) is a p order 
polynomial with z-transform 

A(z) = 1-1- aiz~^ -f- (I 2 Z ^ -f . + dpZ ^ 
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The order, jp, is called the LPC analysis order Thus the output of the 
LPC spectral analysis block is a vector of coefficients (LPC parameters) 
that specify (parametrically) the spectrum of the all-pole model that best 
matches the signal spectrum over the period of time in which the frame of 
speech samples was accumulated 

In the next sections we will be describing m brief both of these signal- 
processing front-ends feature processors The mathematical details and deriva- 
tions will be omitted here, for more comprehensive description the interested 
reader is referred to [19] 

3.1 LPC Front-End Feature Processor 

The theory of linear prediction, as applied to speech, has been well under- 
stood for many years [16]. The LPC front end feature has been widely used 
in speech recognition systems Figure 3 3 shows a block diagram of the LPC 
feature processor. The basic steps required in this feature processing include 
the following 

1 Preemphasis. The digitized speech signal s{n), is put through a low- 
order digital system, to spectrally flatten the signal and to make it 
less susceptible to finite precision effects later in the signal processing 
The digital system used in the preemphasizer is either fixed or slowly 


N M 



Figure 3 2. LPC analysis model 
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adaptive (e g , to average transmission conditions, noise backgrounds or 
even to average signal spectrum) The most widely used preemphasis 
network is the fixed first-order system, 

H{z) = 1 — az~^, 0 9 < a < 1 0 

2 Frame Blocking In this step the preemphasised speech signal, s(n), 
is blocked into frames of N samples, with adjacent frames being sep- 
arated by M samples When M < N, then the adjacent frames will 
overlap, and the resulting LPC spectral estimates will be correlated 
from frame to frame, li M N, then LPC spectrum estimates will be 
quite smooth On the other hand, if M > N, there will be no overlap 
between adjacent frames, in fact some of the speech will be totally lost 
and the correlation between the resulting LPC spectral estimates of 
adjacent frames will contain a noisy component whose magnitude will 
increases as M increases This situation is intolerable in any practi- 
cal LPC analysis for speech recognition If we denote the frame of 
speech by xi{n), and there are L frames within the entire speech signal, 
then 


xi{n) = s{Ml + n), n = 0, 1, ,A^ — 1, / = 0, 1, ,L— 1 

3 Windowing. The next step in the processing is to window each individ- 
ual frame so as to minimize the signal discontinuities at the beginning 
and the end of each frame by tapenng the signal to zero at the beginning 
and end of each frame If we define the window as w{n), 0 < n < N—1, 

then the result of windowing is the signal 

i/(n) = xi(n)w(n), 0 < n < N - 1 

A typical window used for the autocorrelation method of LPC is the 
Hamming window, which has the form 

(2i7rTi \ 

0<n<N-l 
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Figure 3 3 Block diagram of LPC processor for speech recognition 

4 Autocorrelation Analysis Each frame of windowed signal is then auto- 
correlated to give 

ri(m) = Xi(n)xi(n-+m), m = 0,l, ,p 

1712=0 

where the highest autocorrelation value, p, is the order of the LPC 
analysis. Typically values of p from 8 to 16 have been used The side 
benefit of the autocorrelation analysis is that the zeroth autocorrela- 
tion, n(0), IS the energy of the frame The frame energy is the 
important parameter for the speech-detection systems 

5. LPC Analysis The next processing step is the LPC analysis which 
converts each frame of p -I- 1 autocorrelations into an “LPC parameter 
set”, in which the set might be the LPC coefficients, the reflection (or 
PARCOR) coefficients, the log area ratio coefficients, the cepstral co- 
efficients, or any other desired transformation of the above sets. The 
formal method for convertmg from autocorrelation coefficients to an 
LPC parameter set is known as Durbin’s method and can be formally 
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given as the following algorithm (here the subscript I on r/(m) is omit- 
ted for convenience) 

E(°)=r(0) 

= {r(t) - l<t<p 

= kt 

Qij‘^ = 0:^*“^^ - fc, a\zl 

JE;W = (1 - 

The above set of equations are solved recursively for i = 1, 2, ,p, and 

the final solution is given as 

Cm ~ LPC coefficients=a^\ 1 < ’tT' < P 

km = PARCOR coefficients 

dm = log area ratio coefficients=log ( 

6. Conversion to Cepstral Coefficients A very important LPC parameter 
set, which can be derived from the LPC coefficient set, is the LPC 
cepstral coefficients, c(m) The recursion used is 

Co == IncT^ 

Cm = flm + Er=/(^)cjfcOm-ib, l<m<p 

Cm ~ Ylkszl (j^)ci;nm-it) 771 > p 

where is the gain term m the LPC model The cepstral coefficients, 
which are the coefficients of the Fourier transform representation of 
the log magnitude spectrum, have shown to be a more robust, reliable 
feature set for speech recognition than the LPC coefficients and its 
other variants 

7 Parameter Weighting Since the low-order cepstral coefficients are sen- 
sitive to overall slope and the higher-order cepstral coefficients are sensi- 
tive to noise, it has become a standard technique to weight the cepstral 
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coefficients by a tapered window so as to minimize these sensitivities 

[9] 


8 Temporal Cepstral Derivative The cepstral representation of the speech 
spectrum provides a good representation of the local spectral proper- 
ties of the signal for the given analysis frame An improved representa- 
tion can be obtained by extending the analysis to include information 
about the temporal cepstral derivative [6] The time derivative of the 
log magnitude spectrum has a Fourier series representation of the form 

|llog|S(exp^“.t)|)= f 

dt m=-oo ^ 

It IS well known that since Cm{t) is a discrete time representation (where 
t IS the frame index), simply using a first-or second-order difference is 
inappropriate to approximate the derivative as its very noisy Hence a 
reasonable compromise is to approximate dcm{t)/dt by an orthogonal 
polynomial fit over a finite length window, that is, 

= ACn,(t) p 5] kCm{t + k) 

01 k=-K 

where p. is an appropriate normalization constant and {2K -f 1) is the 
number of frames over which the computation is performed 


3.2 Bank-of-Filter Front-End Processor 

The bank-of-filter front-end processors can be broadly classified into two 
categories depending upon the type of filter bank used There are mainly 
two types of filter bank used for speech recognition, namely the uniform filter 
bank and non-uniform filter bank. 

In uniform filter bank, all filter covering the frequency range of speech, 
are uniformallv spaced. On the other hand the non-uniform filter bank is 
designed accorffing to some criterion for how the individual filters should 
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be spaced in frequency One commonly used criterion is to space the filters 
unifoimally along the logarithmic frequency scale which is often justified from 
a human auditor} perception point of view 

An alternatne criterion for designing a non-uniform filter bank is to use 
the critical band scale directly The spacing of the filters along the critical 
band is based on the perceptual studies and is intended to choose bands 
that give equal contribution to speech articulation The general shape of 
the critical band scale is given m Figure 3 4 The scale is close to linear for 
frequencies bolov about lOOOJy^; (i e , the bandwidth is essentially a constant 
as a function of /), and is close to logarithmic for frequencies above lOOOifz 
(i e , the bandwidth is essentially exponential as a function of /) 



Figure 3 4: Critical band scale 


A sight variation of this cntical band scale, the MEL-scale, the most 
widely used and studied filter bank methods For this work we have also 
used MEL-scale based filter bank feature. Figure 3 5 shows a block diagram 
of this front end feature processor The basic steps required m this feature 
processing are as follows 

1 Preemphasts, Frame Blocking, and Windowing These first three steps 
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FILTER COEFHCIENTS 


Figure 3 5. Block diagram of Mel-scale filter bank processor for speech recog- 
nition 


are essentially the same as in the LPC processor and have been ex- 
plained in detail in the LPC section earlier 

2 Calculation of Energy Vector Each of the windowed waveform seg- 
ments is transformed into the frequency domain by computing the FFT 
of the corresponding waveform A vector of log energies is then com- 
puted from each waveform segment by weighting the FFT coefficients 
by the magnitude frequency response of the filter bank The log ener- 
gies are taken for the purpose of dynamic range compression and also 
in order to make the statistics of the estimated speech power spectrum 
approximately Gaussian 

3 Computing DCT. The final processing stage is to apply the discrete 
cosine transform (DCT) to the log energy coeflficients This has the ef- 
fect of compressing the spectral information into the lower-order coeffi- 
cients, and it also decorrelates them to allow the subsequent statistical 
modelmg to use diagonal covanance matrices 
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Chapter 4 

Database and Implementation 


4.1 Database 

For this task we have used “30k Numbers corpus” obtained from Oregon 
Graduate Institute (OGI) The numbers corpus is a collection of ordinal and 
cardinal numbers, continuous digit strings and isolated digit strings The 
utterances were taken from numerous telephone speech data collection efforts 
done by Center for Spoken Language Understanding (CSLU) at OGI This 
corpus contains 15,000 files. Each file has an orthographic transcription; 
about 7000 have a phonetic transcription The speech samples m database 
are sampled at 8 kHz 

In our study, only the utterances which contain digits are used and the 
collection of these utterances will now be called as digit corpus The digit 
corpus IS divided into three sets, namely training, development and test sets. 
In our study, training set consists of 3/5th of digit corpus while the share of 
development and test sets are l/5th each The development set is further 
divided into four sub-sets, namely Dev, Devi, Dev2, and Dev3 The set Dev 
contains all files that have phonetic labels. The other three sub-sets are ob- 
tained by splitting up all available developments files mto three parts such 
that these resulting sub-sets are speaker- independent. In this study we have 
mainly used training set for model training and Devi for testing 
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4.2 Implementation 

We have made use of “CSLU Speech Toolkit” which apart from many other 
utilities for speech processing also provides the development environment for 
the hidden Markov modeling based speech recognizers. This toolkit may be 
downloaded from http //www cse ogi edu/CSLU/toolkit 

Defining the task 

The grammar used by our continuous digit recognizer is depicted in Fig- 
ure 4 1 This grammar can be used to recognize any number of spoken 
digits, with optional silence between words 



Figure 4 1 Continuous digit string grammar 
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Model Prototyping 


The word pronunciation models for the words defined by the grammar are 
given in Table 4 1 These are defined using the ARPA phonetic alphabet 
Based on the pronunciation models we define the first set of HMM models 
required for the task 


one 

w ah n 

seven 

s eh V ah n 

two 

t uw 

eight 

ey t 

three 

th r ly 

nine 

n ay n 

four 

f aor 

zero 

z ih r ow 

five 

f ay V 

oh 

ow 

SIX 

s ih k s 

sil 

Sll 


Table 4 1 Word Pronunciations for the Digit Recognizer 


Feature extraction 

We have used two types of features in this work, namely LPC-cepstral coeflSl- 
cients and MEL-cepstral coefficients The different parameters settings used 
LPC- and MEL-cepstral feature extraction processes are given in Table 4 2 
and Table 4 3 respectively In both cases along with base features, the first 
and second order time derivatives were also computed The time derivatives 
were computed over 5 frames Thus the size of feature vector for each frame 
is 36 m case of LPC and 39 for MEL We have also done cepstral mean sub- 
traction in both cases 
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Window size 

16 ms 

Frame size 

10 ms 

Data preemphasis 

0 98 

Analysis order 

10 

No of cepstral coeff 

12 

Cepstral liftering coeff 

06 


Table 4 2 LPC-Cepstral Feature Generation Parameter Setting 


Wmdow size 

16 ms 

Frame size 

10 ms 

Data preemphasis 

0 98 

No of filters 

21 

No of cepstral coeff 

13 

Cepstral liftering coeff 

06 


Table 4 3 MEL-Cepstral Feature Generation Parameter Setting 
Model Training 

The first step to tram the chosen mono phone models is to initialize them 
The initialization is a three step process First, all data (segments) for the 
particular class are loaded into the memory Each segment is then cut into 
equal sized segments, depending upon the number of states in the particular 
model All data allocated to a particular HMM state is then combined and 
the initial mixture mean vectors were estimated using vector quantization 
The initial mixture variances were set to the pool variance of the data During 
this step mixture weights and state transition probability matrices were not 
computed. After that the parameter estimates were further improved by 
Viterbi state ahgnment Fmalh. the model parameters were estimated using 
expectation /maximization (EM) algonthm 
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Allophonic variations 

The seed models developed in previous step do not distinguish same sounds 
which are produced in different parts of a word For instance, the “t” in two 
will be pronounced differently from the “t” m eight In this case the burst 
of sound “t” m eight is sometimes unreleased, and therefore needs to be 
modeled differently This can be done by cloning the model “t”, but instead 
of using standard left-to-nght model, this third state is made optional 
For the digits which end in nasal (one/seven/nine) the speaker often tend 
to emphasize the word’s final phoneme These variations are captured by 
cloning the <ah> model Table 4 4 hsts the new set of pronunciations given 
the design considerations discussed 


one 

w ah n [ah3] 

seven 

s eh V ah2 n [ah3] 

two 

t uw 

eight 

ey td 

three 

th r ly 

nine 

n ay n 

four 

f aor 

zero 

z ih r ow 

five 

f ay V 

oh 

ow 

SIX 

s ih ks 

Sll 

Sll 


Table 4 4 Word Pronunciations for the Digit Recognizer based on Allophonic 
Variations 


Transcription 

The phonetically hand labeled data are typically only sufficient to create 
“seed” models for a phoneme-based recognizer To build a more accurate 
and more robust recognizer requires much more data Most databases con- 
tain word level transcription, which may be used to augment the existing 
training data For that a finite state grammar is created using input word 
transcription where each node or state in the grammar contains a word and 
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its pronunciation variants obtained by a pronunciation lexicon The stan- 
dard Viterbi algorithm is then used to find the best possible path through 
the grammar which results in the selection of the pronunciation variants to 
fit the sentence 

The resulting output word transcription may then be used to generate 
the associated model transcription for HMM embedded training 

Embedded model re-estimation 

Model initialization and single model training only use the data associated 
with the particular model During the training step it is assumed that the 
phonetic boundaries are defined and there is no interaction between neigh- 
boring models This problem is addressed m embedded parameter estimation 
step by creating a composite model by combining the models defined by the 
word transcription file for each of the training utterances specified Training 
then continues similar to the single model training Using the composite 
model, however, the results in all models updated simultaneously 

Evaluation 

The previous training steps creates a series of HMM models Now the model, 
which gives best performance when evaluated on previously unseen data, is 
selected To setup the evaluation process we first need to construct a task 
grammar and associated search network Then a Viterbi decoder searches 
through the finite state grammar to give recognition answers which are com- 
pared with input transcriptions to perform the scoring All extraneous speech 
labels are also suppressed during the scoring 

The outline of the typical process of training an HMM model for speech 
recognition is shown in Figure 4 2. 
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Chapter 5 

Experimental Results 


5.1 Results 

We have impleineutecl a continuous digit llhIM iecogiuz;ci and its jieiloi- 
inance is evaluated for LPC and MEL fiont-end features with diffciciit model 
size (number of states) and number of mixtuies per state The peifoimance 
(accur£|fies) of the recognition system for both LPC and MEL fcatuies aie 
given in Table 5 1 and Table 5 2 lespectnely In these tables two accuiacies 
are given corresponding to a jiaitu ular nuinbei of state and a paitieuhu uum- 
bei ol mixtuie pei state, the top one is woid accuiacy and the bottom one 
is sentence accuracy The plots of woid lecognition accuiacy as a function 
of the number of mixtures per state m the model for models with numbei 
of states 1, 3, 4, and 5 aie given m Figiiie 5 1, Figuie 5 2, Figuic 5 3, 
and Figure 5 4 respectively The plot of coiiespondmg sentence leiognition 
accuracies is given in Figuic 5 5 

The salient infeiences dciived fiom this study aie as follows 

• The recognition accuracy with MEL-ccpstial feature is better than with 
LPC-cepstral feature This impiovcment is in between 0 5-1% with 
simple phoneme/ word grammar 

• In case of digit recognition a woid accuiacy below 94% is fai fiom 
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satisfactory, thciefore, our study indicates that a product of the number 
of states and the number of mixtures per state in a model below ten is 
inadequate for satisfactory performance 

• The single mixture (multivariate normal density) is inadequate to rep- 
resent speech parameter satisfactorily, this is in accordance with the 
results reported by Rabiner et al [20] 

• Comparing the word accuracy with LPC- and MEL-cepstral features 
particularly at lower mixtures per state, one may notice that perfor- 
mance of LPC feature is better than that of MEL feature These results 
provide some indication that speech spectrum with LPC feature is less 
multi-modal than with MEL feature and also that MEL feature is more 
descriptive than LPC feature that is why it supersedes the LPC one at 
higher mixtures per state 

• Comparing the sentence accuracy we found that the performance with 
MEL feature is also better than LPC feature on an average by 1% 

• We also noticed that for a fixed product of the number of states and 
the number of mixtures per state in a model, models having higher 
number of states provide better word and as veil as sentence accuracy 
than those containing less number of states This improvement is more 
pronounced when the product of the number of state and number of 
mixture per state in a model is below twelve 

5.2 Conclusion 

Our study indicates that MEL-cepstral feature performs better than LPC- 
cepstral feature though this improvement is not very significant with a model 
of sufiiciently large size (number of states and number of mixtures per state) 
This stud} also mdicates that a model size (4 states and 3 mixture per state) 
with MEL-cepstral feature is good enough for continuous digit recognition 
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task as it provides sufficiently high accuracy (95%) with moderate model 
complexity 


No. of 

States 

No. of Mixtures ] 

per State 

1 

2 

3 

4 

5 

1 

53 03% 
18 22% 

75 07% 
33 10% 


83 07% 
44 50% 


2 





91 17% 
71 07% 

3 

87 98% 
58 41% 

90 0% 
72 18% 


93 86% 
78 02% 

i 

4 



94 29% 
79 69% 


94 99% 
82 75% 

5 

91 14% 
67 59% 

94 03% 
78 44% 


95 13% 
82 06% 

1 


Table 5 1 Recognition Accuracies (Word &: Sentence) with LPC Feature for 
Models with Different States and Mixtures per State 
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No. of 

States 

No. of Mixtures per State 

1 

2 

3 

4 

5 

1 

49 35% 
13 35% 

74 61% 
28 79% 


85 19% 
48 96% 


2 






3 

86 44% 
53 41% 

92 98% 
73 71% 


94 82% 
80 67% 


4 

j 


95 54% 
83 30% 


95 49% 
84 01% 

5 


94 36% 
79 28% 


95 59% 
83 45% 



Table 5 2 Recognition Accuracies (Word & Sentence) with MEL Feature for 
Models w'lth Different States and Mixtures per State 



Figure 5 1 Word recognition accuracy versus the number of mixture per 
state for model with 1 state 
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95 


No of mixture per state 

Figure 5 2 Word recognition accuracy versus the number of mixture per 
state for model with 3 states 


No of mature per state 


Figure 5 3 Word recognition accuracy versus the number of mixture per 
state for model with 4 states 





Figure 5 4 Word recognition accuracy versus the number of mixture per 
state for model with 5 states 



Figure 5 5. Sentence recognition accuracy versus the number of mixture per 
state for models with 1, 3, and 5 states 
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Chapter 6 

Study of Relationship between 
Speakers 

6.1 Motivation 

Since last five decade a lot of leseaich has already been done to solve the 
problnjn of speech iccogiution and a laigc amount of undci standing has been 
developed in this piocess, but the complete solution of the pioblcin still 
remains elusive Thcic aic still some aieas in which the uudoistandmg till 
date IS insufficient sucli as human auditory mechanism and speech pioduction 
process 

One of the inajoi pioblems in speech lecogmtion is variability of speech 
Though human beings are able to handle the vaiiabihty in speech veiy well, 
machines have not been able to oveicome this pioblem lully Thus in oidei 
to undeistand and [laiameteir/e (he vauability, one may get the motivation 
to ask, given any two spcakei what is the lelationship between them m tciins 
of speech related paiarneteis This query not only befits the human natuie of 
enquiry, but is also of great impoitance in the field of speech lecogmtion by 
machine, particularly in the speaker mdoiiendcnt speech iccogiiition systems 
It is well known that dilfeicnt speakers have diflcicnt foimant fiequencies 
for the same vowel [ 18 ] The vowels aie almost completely chaiacteiized 
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by the first three formant (resonance) frequencies of the vocal tract transfer 
function These formant frequencies are m turn affected by the length of the 
pharyngeal-oral tract, the narrowness of the tract, and the constriction along 
the tract Out of these factors, the difference in vocal-tract length for different 
speakers is attributed to be the mam source of variability among different 
speakers The reason behind the above attribution is the gross approximation 
of the vocal-tract of a speaker by a uniform tube of characteristic length, 
so that the resonant frequencies in the speech produced by a speaker are 
inversely proportional to the length of his/her vocal-tract [26] Experimental 
observations have reported the length of vocal-tact typically 18 cm in males 
and 13 cm in females [25] 

The above idea is reported in the literature as “scale relationship” among 
speakers and is demonstrated as a valid model of the speech spectrum [1, 
25] Mam researchers used the scale model of the speech spectrum and 
reported improvement in recognition accuracy by estimating the scale-factor 
and normalizing the spectra of different speakers [4, 10, 14] This is, however, 
an expensne procedure since the scale factor has to be estimated for every 
speaker with respect to reference speaker Recently Umesh et al [23] have 
suggested the use of features based on scale transform that do not require the 
explicit computation of scale factor and avoids the computation complexity 
The basic idea used in all such systems is to derive features which help provide 
insensitiviti to the scale factor existing among the speech spectra of different 
speakers Such scale invariant features are obtained through the use of some 
suitable transforms viz, Scale-transform, Mellin-transform [3] 

Even with this much understanding about the relationship among differ- 
ent speakers, the problem of speaker variability is still not completely solved 
A recent study about scale relationship has reported the frequency depen- 
dence of the scale factor relating different speakers in scale relationship [22] 
Thus scale model of the speech spectrum is an approximation of the ideal 
model and further research is necessary to identify this ideal model 

As a first step towards this goal, we have tried to see that if there is some 
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simple relationship between spectra of any two speakers In trying to ‘dentify 
this simple relationship, we have looked at the large class of simple models 
and have not been biased by the physiological considerations or otherwise 


6.2 Database 

We have used a database obtained by Gordon Peterson and Harold Barney 
[18] It contains the values of first three formant frequencies for 10 vowels 
/lY/, /IH/, /EH/, AE/, /AH/, /AA/, /AO/, /UH/, /UW/, and /ER/, de- 
rived from the speech collected from 33 male, 28 female and 15 child speakers 
For our study, we have divided the above database into three subsets, namely 
DBl, DB2, and DB3 Subset DBl consists of 8 male speakers, 8 female speak- 
ers and 8 child speakers while subset DB2 also contains same number of male, 
female, and child speakers but these are different from DBl except one child 
speaker which common to both sets as there were only 15 child speakers in 
the database Subset DB3 is the same as DBl except for all speakers only 
first two formant frequencies are considered 


6.3 Method of Search 

Since we had no idea about the functional form and the number of parameters 
in the probable relationship among speakers at the outset We even didn’t 
know there at all exists a unique relationship among speakers or not Thus 
we had extremely big task in hand Therefore to limit the task, we had 
decided to search for only simple relationship and that too not containing 
more than four parameters Some justification for such limited search can be 
derived from the two facts First, a relationship with higher parameter count 
is more likely to fit the experimental data than the one with lower parameter 
count and secondly it would be extremely difficult to provide some physical 
explanation to a fancy relationship 

To search for the best fit simple relationships to our speech data we have 
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used a curve fitting software package named “TableCurve 2D v4 0” In our 
search we have tried to fit the simple curves only to the data which is the 
collection of the values of the formant frequencies for 10 vowels There were 
two set of values for all formants per speaker Thus we have formed a vector 
of 60 elements (10 vowels x 3 formants x 2 examples) in case of datasets 
DBl and DB2 while a vector of 40 elements in case of dataset DBS as only 
first two formants were considered 

The procedure followed the search is summarized as follows 

• In every dataset, we first took first four male speakers and first four 
female speakers, then every male speaker was associated once with all 
the four female speakers and the lists of fit equations were noted in 
each case 

• The coefficient of least square error was used as the criterion for finding 
the fit simple equations in each case 

• In all cases only those resulting fit equations were considered in the 
study, for which the coefficient of least square error was not below 
three percent of the error coefficient of the best fit equation 

• Then we took the remaining four male speakers and first four child 
speakers and associated them to in the same manner as explained pre- 
viously and the fit equations were noted as per above laid criteria 

• After that the remaining four female speakers were associated with the 
remaining four child speakers were associated and fit equations were 
noted similarly as above 

• In this way we had noted 48 different lists of fit simple equations We 
had also reversed the association m each of the above cases Therefore 
we finally had 96 different lists of fit equations with each datasets 

Following the above procedures we are left with 20-30 fit equations in 
each case. Now to narrow down our search further we have followed certain 
thumb rules to attach score to any particular fit equation 
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Firstly, if the coefficient of least square error of any equation in short- 
list of fit equations is equal up to two decimal places to that of the best fit 
equation in that short-list , a score of 1 is awarded and if its error coefficient 
IS greater than bj one at the second place from decimal than that of best fit 
one, a score of 2 is given and so on 

Secondl}, all those fit equations, which are equal up to two decimal places, 
are further segregated by adding 0 5 to their score due to first rule depending 
on whether their error coefficient is above the half of the range or not 

In this way we found the associated score with every equation in the 
short-listed fit equations in each case and then for every dataset few best fit 
equation are listed based on their total accumulated score The lower the 
score the better fit that equation for a particular dataset 

6.4 Results 

We have searched for the fit simple equations m the manner explained m the 
previous section The list of the top 15 fit equations for all the three datasets 
are given m Table 6 1 In this table the fit equations are expressed in terms 
of X and y parameteis which represent the formant frequencies of any two 
chosen speakers 

Following inferences can be deduced from our study 

• This stud} indicates that the equation {y — ax^) figures out to be the 
best relationship among speakers 

• The scale relationship (y = ax), w’hich is well accepted and possesses 
physical explanation, is the special case of the equation(y = ax'’) 

• This stud} also validates the scale-relationship among speakers as it 
appeared among the top ten relationships in all datasets 

• The equation {y — a + bx) and those resembling its form have showm 
better performance than those highly different in form This provides 
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Fit Equation 
Formula 

Relative Order of Fit 

DBl 

DB2 

DBS 

y = ax^ 

I 

I 

I 

y = a-\-b exp~^'''^ 

II 

I 

I 

y = a + bx/hxx 

II 

II 

I 

yO-5 = 0 + 53-0 5 

II 

II 

II 

y = a + bx 

III 

II 

I 

In y = c + 5/ In T 

III 

III 

IV 

y = a + 6x In a: 

III 

III 

II 

In y = G + 6 In T 

I\^ 

III 

III 

yO-5 = a + 5(ln.T)^ 

I\^ 

IV 

III 

y° ® = G + ® In X 

IV 

IV 

II 

y ax 

IV 

IV 

III 

y = a + bx^ 

V 

V 

I 

y^ = G + 

\' 

V 

II 

y^ = a + In x 

\' 

V 

IV 

y~^ = a + b/x 

\’I 

V 

IV 


Table 6.1; List of fit equations for datasets DBl, DB2, and DB3 


some clue that the ultimate relation may be close in form to the linear 
transformation. 

• Comparing the performance of different fit equations across all the three 
datasets one may easily notice that all most all listed equations have 
relatively consistence except equation {y = a+bx^) which has suddenly 
jumped to order I in case of dataset DBS. The one possible reason 
that we could able to point out. is the variability of the third formant 
value in the database. The first two formant values exhibit significantly 
less variability and he mostly along the curve exhibited by equation 
(y = a + bx^). 
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6.5 Conclusion 


In this study we have tried to narrow down the choice of the probable 
relationship based on formant frequencies between any two speakers. Such a 
study might provide clues in finding appropiate transforms to remove speaker 
dependencies and improve speaker independent speech recognition. A future 
study using the whole foimant envelopes might throw more light and in turn 
provide more reliable relationships. 
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